f-tests and their applications 🐟

---

layout: true
  
<div class="my-footer">
<span>
<a href="https://psychmethods.github.io/coursenotes/" target="_blank">Methods in Psychological Research</a>
</span>
</div>

---

# f-tests and their applications

---

## Roadmap: Last Week 
.large[
- t-tests
- t-tests
- t-tests
- confidence intervals
]

---

## Roadmap: This Week
.large[
- multiple comparisons
- f-test logic
  - analysis of variance
  - regression
]

---

# Context

---

# So far...

- We have learned how to test hypotheses using the `$z$`- and `$t$`-distributions
  - With 1 or 2 tails
  
--
  - With 1 or 2 groups
  
--
  - With 1, 2, or Paired Samples

--
- But what about more than two groups?

---

## analysis of variance (ANOVA)

- Is method for comparing any number of groups.
- It can be thought of as an extended multi-group t-test 
  - OR as a regression with grouping variables
- Is an efficient alternative to running n t-tests to compare n-groups

---

## Why can't we Just Run a Bunch of t-tests?

- For example, in a three group design, we could just run 3 t-tests?
--

- Problems: 
  - Many t-tests with lots of groups
  - Alpha-control
  
---

## Alpha-control

- Increase type-1 error rate (because we're testing multiple times)
- Multiplication of type 1 errors
  - e.g, If we test 20 times, we'll find 1 significant result by chance (if `$\alpha$` = .05)
- http://shinyapps.org/apps/p-hacker/
- ANOVA controls alpha =.05 across all tests (as does regression)

---

## Consequences of Inflated Type 1 Errors

- False Positives (Failure to Replicate)
- Remember that a p-value indicates the probability that the result occurred under the null hypothesis.

--
- If we run many tests, some will be significant by chance alone.

--
- 'Replication Crisis'

---

## Replication Crisis

.pull-left[
- Scientists have found that the results of many scientific studies are 
  - difficult or impossible to replicate on subsequent investigation, 
  - either by independent researchers or 
  - by the original researchers themselves.
- In the early 2010s, as part of a growing awareness of the problem, phenomenon was deemed a "crisis"
- Particularly in social psychology and medicine.
]
--
.pull-right[
<img src="data:image/png;base64,#../img/replicationcrisis.png" width="75%" style="display: block; margin: auto;" />
.footnote[Source: https://xkcd.com/3117/]
]

---

## Attempt to Replicate Psychology

- A report by the Open Science Collaboration in August 2015 
  - estimated the reproducibility of 100 studies in psychological science from three high-ranking psychology journals.
- Overall, 36% of the replications yielded significant findings (p-value below .05) 
  - compared to 97% of the original studies that had significant effects.
- The mean effect size in the replications was approximately half the magnitude of the effects reported in the original studies.

---

---

## Attempt to Replicate Psychology

- Replication rates varied by subfield and journal
- Studies in the field of cognitive psychology had a higher replication rate (50%) than studies in the field of social psychology (25%).
- By Journal -- study replication rates were... 
  - 23% for Journal of Personality and Social Psychology, 
  - 38% for Journal of Experimental Psychology: Learning, Memory, and Cognition; and 
  - 38% for Psychological Science.

---

## Issue is broader than psychology

- At the time, a 2016 poll of 1,500 scientists reported in Nature, 
  - 70% of them had failed to replicate at least one other scientist's experiment (50% had failed to replicate one of their own experiments). 
- Results differed by field
  - chemistry: 90% (60%),
  - biology: 80% (60%),
  - physics and engineering: 70% (50%),
  - medicine: 70% (60%),
  - earth and environment science: 60% (40%).

---

# Wrapping up...

---

# Back to ANOVA...

---

## Multi-group tests can be conducted multiple ways
- I'm going to show you two popular approaches (regression & ANOVA).
- I was taught it using regression, so I am biased. 
- But the majority of psychologists are taught using the traditional method of ANOVA.
  - Regardless, it is useful to be familiar with that approach.

---

# Traditional Approach
## Analysis of Variance (ANOVA)

---

## ANOVA Logic

- Analysis of Variance (ANOVA)
  - is a statistical method used to test differences between two or more means.
- Why call it "Analysis of Variance" rather than "Analysis of Means"? 
  - The inferences about means are made by analyzing variance.
  - Technically, compares variance between groups to variance within groups
- ANOVA is used to test general rather than specific differences among means. 
  - e.g., Are there any differences among the group means?
---

## Goals of ANOVA

- Conceptually, the goal of ANOVA is to 
  - determine the amount of variability in groups of data, and 
  - to see if the variability is greater between groups than within groups.
--

- ANOVA is a more general version of the t-test
  - Both tests use sample data to test hypotheses about population means.
--

- ANOVA can test hypotheses about two or more population means. 
  - The T-Test can only test hypotheses about two population means.
  - T-Test can only be used with one independent (classification) variable, whereas 
  - ANOVA can be used with more any number of independent (classification) variables.
---

## Variability in ANOVA
- In ANOVA, we analyze two types of variability:
  - Variability **between groups**: Differences due to the interaction between the groups (treatments)
  - Variability **within groups**: Differences due to random chance (error)

---

## F-Ratio
- The F-Ratio is the ratio of the variance between groups to the variance within groups.
- If the null hypothesis is true (all group means are equal),
  - the variance between groups should be similar to the variance within groups,
  - and the F-Ratio should be close to 1.
--

- If the null hypothesis is false (at least one group mean is different),
  - the variance between groups should be larger than the variance within groups,
  - and the F-Ratio should be greater than 1.

---

## F-Distribution

.pull-left[
<img src="data:image/png;base64,#../img/fdist.png" width="95%" height="95%" style="display: block; margin: auto;" />

]

.pull-right[
- Looks like a skewed normal for many combinations of parameters
- Has two parameters
  - Degrees of freedom in the numerator
  - Degrees of Freedom in the denominator
  - Critical Value is in the tail
      - All ANOVAs are one-tailed tests
      - In the Upper Tail
]
---

## F-Ratio Formula

- The F-Ratio is calculated using the following formula:
`$$F = \frac{MS_{between}}{MS_{within}}$$`
- Where:
  - `$MS_{between}$` is the mean square between groups (variance between groups)
  - `$MS_{within}$` is the mean square within groups (variance within groups)
---

## Degrees of Freedom
- The degrees of freedom for the F-Ratio are determined by:
  - Numerator degrees of freedom (`$df_{between}$`): Number of groups - 1
  - Denominator degrees of freedom (`$df_{within}$`): Total number of observations - Number of groups

--
- These degrees of freedom are used to determine the critical value of F from the F-distribution
  - to assess the statistical significance of the F-Ratio.
---

# Wrapping Up...

---

# Implementing an F-test

---

## Recall: General Procedure

.large[
1. Decide what type of test we want to use
2. Decide what the null and alternative hypothesis is.
3. List what we have
4. Compute test statistic (F-statistic)
5. finding critical value
6. Compare test statistic to critical value
  - (we can also calculate p and compare to `$\alpha$`)
7. Make decision: reject null or not, and draw conclusion
]

---

# Example: Comparing tropical flowers

- On the island of Dominica, we’re going to examine the relationship between 
  - varieties of the tropical flower Heliconia and the 
  - different species of hummingbirds that fertilize them. 
- Researchers wondered if
  - the lengths of the flowers and 
  - the forms of the hummingbirds' beaks have evolved to match each other. 
---

# Example: Comparing tropical flowers
.pull-left[
- They measured the flower lengths (in mm) of three different species of Heliconia flowers.
- The table gives length measurements (in millimeters) for samples of three varieties of Heliconia, each fertilized by a different species of hummingbird. 
- Do the three varieties display distinct distributions of length? 
- Are the mean lengths of their flowers different?
]

.pull-right[
<div id="pzveloiefp" style="padding-left:0px;padding-right:0px;padding-top:10px;padding-bottom:10px;overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>#pzveloiefp table {
  font-family: system-ui, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol', 'Noto Color Emoji';
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
}

#pzveloiefp thead, #pzveloiefp tbody, #pzveloiefp tfoot, #pzveloiefp tr, #pzveloiefp td, #pzveloiefp th {
  border-style: none;
}

#pzveloiefp p {
  margin: 0;
  padding: 0;
}

#pzveloiefp .gt_table {
  display: table;
  border-collapse: collapse;
  line-height: normal;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 12px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#pzveloiefp .gt_caption {
  padding-top: 4px;
  padding-bottom: 4px;
}

#pzveloiefp .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#pzveloiefp .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 3px;
  padding-bottom: 5px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#pzveloiefp .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#pzveloiefp .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#pzveloiefp .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#pzveloiefp .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#pzveloiefp .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#pzveloiefp .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#pzveloiefp .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#pzveloiefp .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#pzveloiefp .gt_spanner_row {
  border-bottom-style: hidden;
}

#pzveloiefp .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  text-align: left;
}

#pzveloiefp .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#pzveloiefp .gt_from_md > :first-child {
  margin-top: 0;
}

#pzveloiefp .gt_from_md > :last-child {
  margin-bottom: 0;
}

#pzveloiefp .gt_row {
  padding-top: 2px;
  padding-bottom: 2px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#pzveloiefp .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#pzveloiefp .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#pzveloiefp .gt_row_group_first td {
  border-top-width: 2px;
}

#pzveloiefp .gt_row_group_first th {
  border-top-width: 2px;
}

#pzveloiefp .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#pzveloiefp .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#pzveloiefp .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#pzveloiefp .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#pzveloiefp .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#pzveloiefp .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#pzveloiefp .gt_last_grand_summary_row_top {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: double;
  border-bottom-width: 6px;
  border-bottom-color: #D3D3D3;
}

#pzveloiefp .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#pzveloiefp .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#pzveloiefp .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#pzveloiefp .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#pzveloiefp .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#pzveloiefp .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#pzveloiefp .gt_left {
  text-align: left;
}

#pzveloiefp .gt_center {
  text-align: center;
}

#pzveloiefp .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#pzveloiefp .gt_font_normal {
  font-weight: normal;
}

#pzveloiefp .gt_font_bold {
  font-weight: bold;
}

#pzveloiefp .gt_font_italic {
  font-style: italic;
}

#pzveloiefp .gt_super {
  font-size: 65%;
}

#pzveloiefp .gt_footnote_marks {
  font-size: 75%;
  vertical-align: 0.4em;
  position: initial;
}

#pzveloiefp .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#pzveloiefp .gt_indent_1 {
  text-indent: 5px;
}

#pzveloiefp .gt_indent_2 {
  text-indent: 10px;
}

#pzveloiefp .gt_indent_3 {
  text-indent: 15px;
}

#pzveloiefp .gt_indent_4 {
  text-indent: 20px;
}

#pzveloiefp .gt_indent_5 {
  text-indent: 25px;
}

#pzveloiefp .katex-display {
  display: inline-flex !important;
  margin-bottom: 0.75em !important;
}

#pzveloiefp div.Reactable > div.rt-table > div.rt-thead > div.rt-tr.rt-tr-group-header > div.rt-th-group:after {
  height: 0px !important;
}
</style>
<table class="gt_table" data-quarto-disable-processing="false" data-quarto-bootstrap="false">
  <thead>
    <tr class="gt_heading">
      <td colspan="9" class="gt_heading gt_title gt_font_normal" style><span class='gt_from_md'>Flower lengths for three Heliconia varieties</span></td>
    </tr>
    <tr class="gt_heading">
      <td colspan="9" class="gt_heading gt_subtitle gt_font_normal gt_bottom_border" style><span class='gt_from_md'>Units: millimeters</span></td>
    </tr>
    <tr class="gt_col_headings">
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="row">row</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c1"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c2"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c3"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c4"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c5"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c6"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c7"> </th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1" scope="col" id="c8"> </th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr class="gt_group_heading_row">
      <th colspan="9" class="gt_group_heading" scope="colgroup" id="H. bihai">H. bihai</th>
    </tr>
    <tr class="gt_row_group_first"><td headers="H. bihai  row" class="gt_row gt_right">1</td>
<td headers="H. bihai  c1" class="gt_row gt_right" style="text-align: center;">47.12</td>
<td headers="H. bihai  c2" class="gt_row gt_right" style="text-align: center;">46.75</td>
<td headers="H. bihai  c3" class="gt_row gt_right" style="text-align: center;">46.81</td>
<td headers="H. bihai  c4" class="gt_row gt_right" style="text-align: center;">47.12</td>
<td headers="H. bihai  c5" class="gt_row gt_right" style="text-align: center;">46.67</td>
<td headers="H. bihai  c6" class="gt_row gt_right" style="text-align: center;">47.43</td>
<td headers="H. bihai  c7" class="gt_row gt_right" style="text-align: center;">46.44</td>
<td headers="H. bihai  c8" class="gt_row gt_right" style="text-align: center;">46.64</td></tr>
    <tr><td headers="H. bihai  row" class="gt_row gt_right">2</td>
<td headers="H. bihai  c1" class="gt_row gt_right" style="text-align: center;">48.07</td>
<td headers="H. bihai  c2" class="gt_row gt_right" style="text-align: center;">48.34</td>
<td headers="H. bihai  c3" class="gt_row gt_right" style="text-align: center;">48.15</td>
<td headers="H. bihai  c4" class="gt_row gt_right" style="text-align: center;">50.26</td>
<td headers="H. bihai  c5" class="gt_row gt_right" style="text-align: center;">50.12</td>
<td headers="H. bihai  c6" class="gt_row gt_right" style="text-align: center;">46.34</td>
<td headers="H. bihai  c7" class="gt_row gt_right" style="text-align: center;">46.94</td>
<td headers="H. bihai  c8" class="gt_row gt_right" style="text-align: center;">48.36</td></tr>
    <tr class="gt_group_heading_row">
      <th colspan="9" class="gt_group_heading" scope="colgroup" id="H. caribaea red">H. caribaea red</th>
    </tr>
    <tr class="gt_row_group_first"><td headers="H. caribaea red  row" class="gt_row gt_right">1</td>
<td headers="H. caribaea red  c1" class="gt_row gt_right" style="text-align: center;">41.90</td>
<td headers="H. caribaea red  c2" class="gt_row gt_right" style="text-align: center;">42.01</td>
<td headers="H. caribaea red  c3" class="gt_row gt_right" style="text-align: center;">41.93</td>
<td headers="H. caribaea red  c4" class="gt_row gt_right" style="text-align: center;">43.09</td>
<td headers="H. caribaea red  c5" class="gt_row gt_right" style="text-align: center;">41.47</td>
<td headers="H. caribaea red  c6" class="gt_row gt_right" style="text-align: center;">41.69</td>
<td headers="H. caribaea red  c7" class="gt_row gt_right" style="text-align: center;">39.78</td>
<td headers="H. caribaea red  c8" class="gt_row gt_right" style="text-align: center;">40.57</td></tr>
    <tr><td headers="H. caribaea red  row" class="gt_row gt_right">2</td>
<td headers="H. caribaea red  c1" class="gt_row gt_right" style="text-align: center;">39.63</td>
<td headers="H. caribaea red  c2" class="gt_row gt_right" style="text-align: center;">42.18</td>
<td headers="H. caribaea red  c3" class="gt_row gt_right" style="text-align: center;">40.66</td>
<td headers="H. caribaea red  c4" class="gt_row gt_right" style="text-align: center;">37.87</td>
<td headers="H. caribaea red  c5" class="gt_row gt_right" style="text-align: center;">39.16</td>
<td headers="H. caribaea red  c6" class="gt_row gt_right" style="text-align: center;">37.40</td>
<td headers="H. caribaea red  c7" class="gt_row gt_right" style="text-align: center;">38.20</td>
<td headers="H. caribaea red  c8" class="gt_row gt_right" style="text-align: center;">38.07</td></tr>
    <tr><td headers="H. caribaea red  row" class="gt_row gt_right">3</td>
<td headers="H. caribaea red  c1" class="gt_row gt_right" style="text-align: center;">38.10</td>
<td headers="H. caribaea red  c2" class="gt_row gt_right" style="text-align: center;">37.97</td>
<td headers="H. caribaea red  c3" class="gt_row gt_right" style="text-align: center;">38.79</td>
<td headers="H. caribaea red  c4" class="gt_row gt_right" style="text-align: center;">38.23</td>
<td headers="H. caribaea red  c5" class="gt_row gt_right" style="text-align: center;">38.87</td>
<td headers="H. caribaea red  c6" class="gt_row gt_right" style="text-align: center;">37.78</td>
<td headers="H. caribaea red  c7" class="gt_row gt_right" style="text-align: center;">38.01</td>
<td headers="H. caribaea red  c8" class="gt_row gt_right" style="text-align: center;">NA</td></tr>
    <tr class="gt_group_heading_row">
      <th colspan="9" class="gt_group_heading" scope="colgroup" id="H. caribaea yellow">H. caribaea yellow</th>
    </tr>
    <tr class="gt_row_group_first"><td headers="H. caribaea yellow  row" class="gt_row gt_right">1</td>
<td headers="H. caribaea yellow  c1" class="gt_row gt_right" style="text-align: center;">36.78</td>
<td headers="H. caribaea yellow  c2" class="gt_row gt_right" style="text-align: center;">37.02</td>
<td headers="H. caribaea yellow  c3" class="gt_row gt_right" style="text-align: center;">36.52</td>
<td headers="H. caribaea yellow  c4" class="gt_row gt_right" style="text-align: center;">36.11</td>
<td headers="H. caribaea yellow  c5" class="gt_row gt_right" style="text-align: center;">36.03</td>
<td headers="H. caribaea yellow  c6" class="gt_row gt_right" style="text-align: center;">35.45</td>
<td headers="H. caribaea yellow  c7" class="gt_row gt_right" style="text-align: center;">38.13</td>
<td headers="H. caribaea yellow  c8" class="gt_row gt_right" style="text-align: center;">37.10</td></tr>
    <tr><td headers="H. caribaea yellow  row" class="gt_row gt_right">2</td>
<td headers="H. caribaea yellow  c1" class="gt_row gt_right" style="text-align: center;">35.17</td>
<td headers="H. caribaea yellow  c2" class="gt_row gt_right" style="text-align: center;">36.82</td>
<td headers="H. caribaea yellow  c3" class="gt_row gt_right" style="text-align: center;">36.66</td>
<td headers="H. caribaea yellow  c4" class="gt_row gt_right" style="text-align: center;">35.68</td>
<td headers="H. caribaea yellow  c5" class="gt_row gt_right" style="text-align: center;">36.03</td>
<td headers="H. caribaea yellow  c6" class="gt_row gt_right" style="text-align: center;">34.57</td>
<td headers="H. caribaea yellow  c7" class="gt_row gt_right" style="text-align: center;">34.63</td>
<td headers="H. caribaea yellow  c8" class="gt_row gt_right" style="text-align: center;">NA</td></tr>
  </tbody>
  
</table>
</div>
]
---

# Example: Comparing tropical flowers

.pull-left[
- Use graphs and numerical descriptions to describe and compare the three distributions of flower length. 
- Finally, ask whether the differences among the mean lengths of the three varieties are statistically significant.

]

Table: Summary statistics for Heliconia lengths

|variety            | sample_size|  mean| median|   sd|  IQR|
|:------------------|-----------:|-----:|------:|----:|----:|
|H. bihai           |          16| 47.60|  47.12| 1.21| 1.47|
|H. caribaea red    |          23| 39.71|  39.16| 1.80| 3.49|
|H. caribaea yellow |          15| 36.18|  36.11| 0.98| 1.23|
]
---

## What do the distributions look like?
.pull-left[
<img src="data:image/png;base64,#../img/histflowers.png" width="95%" height="95%" style="display: block; margin: auto;" />
]

.pull-right[
- The three varieties differ so much in flower length that there is little overlap among them. 
- In particular, the flowers of bihai are longer than either red or yellow. 
- The mean lengths are 
  - 47.6 mm for H. bihai, 
  - 39.7 mm for H. caribaea red, and 
  - 36.2 mm for H. caribaea yellow.
.question[
- Are these observed differences in sample means statistically significant? 
- We must develop a test for comparing more than two population means.
]
]

---

# T-test?: Comparing tropical flowers

- We might consider performing multiple t-tests to compare the three means.
- (Sample) means:
  - bihai: 47.60
  - red: 39.71
  - yellow: 36.18
  
- Null hypothesis: The true means (for length) are the same for all groups (the three flower types).

- We could look at separate t tests to compare each pair of means to see if they are different:
  - bihai vs. red
  - bihai vs. yellow
  - red vs. yellow

- However, this gives rise to the problem of multiple comparisons....

---

# Multiple Comparisons Problem

- Statistical methods for dealing with multiple comparisons usually have two steps:
  - An overall test to see 
    - if there is good evidence of any differences among the parameters that we want to compare.
  - A detailed follow-up analysis
    - to decide which of the parameters differ and
    - to estimate how large the differences are.
---

# Multiple comparisons

- The overall test, though more complex than the tests we have met to this point, 
  - is reasonably straightforward. 
- Formal follow-up analysis can be quite elaborate. 
- We will concentrate on the overall test and 
  - use data analysis to describe in detail the nature of the differences.

---

# The analysis of variance F test

- We want to test the null hypothesis of no differences among the population means 
- H_0: 𝜇_1=𝜇_2=...=𝜇_k
- The basic conditions for inference are 
  - that we have random samples from the three populations, and 
- that flower lengths (in our example) are normally distributed in each population.

---
# The analysis of variance F test
- The alternative hypothesis is that there is some difference. 
- That is, not all means are equal. 
  - H_a:  "not all of"𝜇_1, 𝜇_2, ..., 𝜇_k  "are equal"
- This alternative hypothesis is not one-sided or two-sided. It is “many-sided.”
This test is called the analysis of variance (ANOVA) F test.

---
# The analysis of variance F test
- It is based on the comparison of two estimates of the common population variance 𝜎
- one estimate derived from the variability within the samples,
- the other from the variability between the sample means.

---

# Return to our example

---

## Step 1: Type of test
.pull-left[
- We have more than two groups (3 varieties of Heliconia)
- We want to compare means across groups
- We will use a one-way ANOVA (one independent variable with multiple levels)
]

.pull-right[
<img src="data:image/png;base64,#01_ftest_roadmap_files/figure-html/unnamed-chunk-10-1.png" width="75%" style="display: block; margin: auto;" />
]
---

## Step 2: Hypotheses
.large[
- Null Hypothesis (`$H_0$`): The mean flower lengths are equal across all three varieties.
  - `$H_0$`: `$\mu_{bihai} = \mu_{caribaea\_red} = \mu_{caribaea\_yellow}$`
--
- Alternative Hypothesis (`$H_1$`): At least one variety has a different mean flower length.
  - `$H_1$`: At least one `$\mu$` is different
]

---

## Step 3: What we have
.large[
- Sample data: Flower lengths for three varieties of Heliconia
- Significance level (`$\alpha$`): 0.05
- Test type: One-way ANOVA
]

---

## Step 4: Compute F-statistic
.large[
- Calculate the F-statistic using the formula:
`$$F = \frac{MS_{between}}{MS_{within}}$$`
- Where:
  - `$MS_{between}$` is the mean square between groups (variance between groups)
  - `$MS_{within}$` is the mean square within groups (variance within groups)
]

``` r
anova_result <- aov(length_mm ~ variety, data = heliconia)
summary(anova_result)
```

```
##             Df Sum Sq Mean Sq F value Pr(>F)    
## variety      2 1082.9   541.4   259.1 <2e-16 ***
## Residuals   51  106.6     2.1                   
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
```
- From the ANOVA table, we find:
  - `$MS_{between}$` = 541.4
  - `$MS_{within}$` = 2.1
- F-statistic = 541.4 / 2.1 = 259.1
---

## Step 5: Find critical value
.large[
- Degrees of freedom:
  - Numerator (`$df_{between}$`): Number of groups - 1 = 3 - 1 = 2
  - Denominator (`$df_{within}$`): Total number of observations - Number of groups = 54 - 3 = 51
- Using an F-distribution table or software, we find the critical value of F at `$\alpha$` = 0.05 with df1 = 2 and df2 = 51.
- Critical value of F ≈ `3.1787993`

]

---

## Step 6: Compare test statistic to critical value

.large[
- F-statistic = 259.1
- Critical value ≈ `3.1787993`
- Since 259.1 > 3ish, we reject the null hypothesis.
]
---
## Step 6: Alternatively, calculate p-value
- Alternatively, we can calculate the p-value from the F-statistic.

``` r
p_value <- pf(259.1, df1=2, df2=51, lower.tail=FALSE)
p_value
```

```
## [1] 1.922139e-27
```

- There is strong evidence that the three varieties of flowers do not have the same mean length. 
- It appears from our preliminary data analysis that bihai flowers are distinctly longer than either red or yellow. 
- Red and yellow are closer together, but the red flowers tend to be longer.

---

# Example Software Output

<img src="data:image/png;base64,#../img/softwareoutput.png" width="55%" style="display: block; margin: auto;" />
---

# Wrapping Up...